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DETAILED ACTION 



Specification 

1. The title of the invention is not descriptive. A new title is required that is clearly 
indicative of the invention to which the claims are directed. 



Claim Rejections - 35 USC § 102 
2. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(a) the invention was known or used by others in this country, or patented or described in a printed publication in this 
or a foreign country, before the invention thereof by the applicant for a patent, 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 351(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 



3. Claims 1, 2, 3, 7, 8 are rejected under 35 U.S.C. 102(b) as being anticipated by Watanabe 
et al (EP 1043689 A2). 

Regarding claim 1, Watanabe teaches an image processing device for determining three- 
dimensional position and/or orientation of an object, comprising: 

image data capturing means for capturing image data containing an image of the object 



(Watanabe: figure 1, numeral 20); 
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model pattern creating means for creating a model pattern based on image data of a 
reference object with a reference orientation relatively to said image capturing means captured 
by said image capturing means, said reference object having a shape substantially identical to 
that of the object (Watanabe: figure 5, numeral 102); 

transformation means for performing two-dimensional and geometrical transformation of 
the created model pattern to generate a transformed model pattern representing an image of the 
object with an orientation different from the reference orientation (Watanabe: figure 5, numeral 
103); 

pattern matching means for performing a pattern matching of the image data of the object 
captured by said image capturing means with the transformed model pattern (Watanabe: figure 6, 
numeral 203); 

selecting means for repeatedly performing the generation of a transformed model pattern 
and the pattern matching of the image data of the object with the transformed model pattern to 
thereby select one of the transformed model patterns in conformity with the image data of the 
object, and obtain information on a position of the image of the object in the image data 
(Watanabe: figure 6, numerals 204-208); and 

determining means for determining three-dimensional position and/or orientation of the 
object based on the information on the position of the image of the object in the image data and 
information on the orientation of the selected one of the transformed model patterns (Watanabe: 
figure 6, numeral 209). 

Regarding claim 2, Watanabe teaches an image processing device for determining three- 
dimensional position and/or orientation of an object, comprising: 
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image data capturing means for capturing image data containing an image of the object 
(Watanabe: figure 1, numeral 20); 

model creating means for creating a model pattern based on image data of a reference 
object with a reference orientation relative to said image data, capturing means captured by said 
image data capturing means, said reference object having a shape substantially identical to that 
of the object (Watanabe: figure 5, numeral 102); 

transformation means for performing two-dimensional and geometrical transformation of 
the created model pattern to generate a plurality of transformed model patterns each representing 
an image of the object with an orientation different from the reference position (Watanabe: figure 
5, numeral 103); 

storage means for storing the plurality of transformed model patterns and information on 
orientations of the respective transformed model patterns (Watanabe: figure 5, numeral 102, 
103); 

pattern matching means for performing pattern matching of the image data of the object 
captured by said image capturing means with the plurality of transformed model patterns to 
thereby select one of the transformed model patterns in conformity with the image data of the 
object (Watanabe: figure 6, numerals 203-208), and obtain information on a position of the 
image of the object in the image data (Watanabe: figure 6, numerals 209); and 

determining means for determining three-dimensional position and/or orientation of the 
object based on information on the position of the image of the object in the image data and the 
information on an orientation of the selected one of the transformed model patterns (Watanabe: 
figure 6, numeral 209). 
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Regarding claim 3, Watanabe teaches a two-dimensional and geometrical transformation 
of an affme transformation ("world coordinate system is obtained by an arithmetic operation of 
coordinate transformation using the data of the position/posture of the workpiece W in the 
camera coordinate system and the position/posture of the camera 20 in the world coordinate 
system"; Watanabe: paragraph [0042]), and said image processing device further comprises 
additional measuring means for obtaining a sign of inclination of the object with respect to said 
image capturing means ("position/posture of the workpiece in the camera coordinate system is 
stored as [xO, yO, zO, aO, PO, 70]c, where a, p, and y mean rotation angle around Y-, Z- axes, 
and *c' means the camera coordinate system"; Watanabe: paragraph [0033]). 

Regarding claim 7, Watanabe teaches a storage means storing an operating orientation of 
the robot relative to the object or storing an operating orientation and an operating position of the 
robot relative to the object ("when a picking-up command is inputted .... three dimensional 
position/posture of the camera 20 on the world coordinate system at this image capturing 
position is outputted to the image processing apparatus 30"; Watanabe: paragraph [0040]); and 
robot control means for determining an operating orientation of the robot or the operating 
orientation and an operating position of the robot based on the determined three-dimensional 
position and/or orientation of the object ("robot controller 10 operates the robot to perform a 
pick-up operation .... based on the three-dimensional position/posture of the workpiece W (Step 
210)"; Watanabe: paragraph [0044]). 

Regarding claim 8, Watanabe teaches an image capturing means mounted on the robot 
(Watanabe: figure 1, numeral 20). 
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Claim Rejections - 35 USC § 103 

4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

5, Claim 4 is rejected under 35 U.S.C. 103(a) as being unpatentable over the combination 
of Watanabe et al (EP 1043689 A2) with Maeda et al (US 2003/0161537 Al), and further in 
view of Inoue (US 2003/0161504 Al). 

Regarding claim 4, Watanabe discloses all elements as mentioned above in claim 3. 
Watanabe does not teach dividing of a model pattern into at least two partial model patterns 
which are subject to the affine transformation to generate transformed partial model patterns, and 
pattern matching of the image data of the object with the transformed partial model patterns to 
determine most conformable sizes, and determines the sign of the inclination based on 
comparison of the sizes of the conformable partial model patterns with each other. 

Maeda teaches a pattern matching of the image data of the object with the transformed 
model patterns to determine most conformable sizes, and determines the sign of the inclination 
based on comparison of the sizes of the conformable model patterns with each other (Maeda: 
figure 1, numeral 32). 

It would have been obvious at the time the invention was made to one of ordinary skill in 
the art to modify the Watanabe reference to determine the sign of the inclination based on 
comparison of the sizes of the conformable partial model patterns with each other as suggested 
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by Maeda, to allow the imaging device to accurately identify "a three-dimensional object using 
images obtained by photographing the object in the various directions" (Maeda: paragraph 
[0004]). 

The Watanabe with Maeda combination as applied above does not teach dividing of a 
model pattern into at least two partial model patterns which are subject to the affine 
transformation to generate transformed partial model patterns. 

Inoue teaches dividing of a model pattern into at least two partial model patterns (Inoue: 
figure 2, numeral 302, 303), which are subject to the affine transformation ("affine 
transformation"; Inoue: paragraph [0107]) to generate transformed partial model patterns. 

It would have been obvious at the time the invention was made to one of ordinary skill in 
the art to modify the Watanabe with Maeda combination as applied above, to utilize partial 
model patterns as suggested by Inoue, to further "correctly classify the input image regardless of 
a fluctuation in illumination, and a state of occlusion" (Inoue: paragraph [0012]). 
6. Claim 5 is rejected under 35 U.S.C. 103(a) as being unpatentable over Watanabe et al 
(EP 1 043689 A2) in view of Okisu et al (US 6,806,903 B 1 ). 

Regarding claim 5, Watanabe discloses all elements as mentioned above in claim 3. 
Watanabe does not teach a measurement of distances from a displacement sensor separately 
provided in the vicinity of said image capturing means to at least two points on the object using 
the displacement sensor, and determines the sign of the inclination based on comparison of the 
measured distances. 

Okisu teaches a measurement of distances from a displacement sensor separately 
provided in the vicinity of said image capturing means to at least two points on the object using 
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the displacement sensor, and determines the sign of the inclination based on comparison of the 
measured distances ("measure distance to two separate portions of an object from the electronic 
camera, and calculate an angle of inclination 9 based on measure two distances"; Okisu: col. 19, 
lines 54-57). 

It would have been obvious at the time the invention was made to one of ordinary skill in 
the art to modify the Watanabe reference to determine the sign of the inclination based on 
comparison of the measured distances as suggested by Okisu, to allow the image processing 
device to properly focus with high degree of accuracy on the targeted object for image capturing. 
7. Claim 6 is rejected under 35 U.S.C. 103(a) as being unpatentable over Watanabe et al 
(EP 1043689 A2) in view of Maeda et al (US 2003/0161537 Al). 

Regarding claim 6, Watanabe discloses all elements as mentioned above in claim 3. 
Watanabe further teaches an additional pattern matching of image data of the object captured 
after said image data capturing means is slightly moved or inclined ("camera may be moved 
parallelly in accordance with the position of the workpiece in the field of view of the camera"; 
Watanabe: paragraph: [0048]). Watanabe does not teach determining the sign of the inclination 
based on judgment whether an inclination of image of the object becomes larger or smaller than 
the selected one of the transformed model patterns. 

Maeda teaches determining the sign of the inclination based on judgment whether an 
inclination of image of the object becomes larger or smaller than the selected one of the 
transformed model patterns (Maeda: figure 1 , numeral 32). 

It would have been obvious at the time the invention was made to one of ordinary skill in 
the art to modify the Watanabe reference to determine the sign of the inclination based on 
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judgment whether an inclination of image of the object becomes larger or smaller than the 
selected one of the transformed model patterns as suggested by Maeda, to allow the imaging 
device to accurately identify "a three-dimensional object using images obtained by 
photographing the object in the various directions" (Maeda: paragraph [0004]). 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Edward Park whose telephone number is (571) 270-1 576. The 
examiner can normally be reached on M-F 10:30 - 20:00, (EST). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Brian Werner can be reached on (571) 272-7401 . The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applicationis 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

Edward Park 
Examiner 
Art Unit 2609 
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